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Analysis  of  the  effectiveness  of  the  national  research  and  development 
effort  directs  attention  to  the  process  of  information  transfer— the  generation^ 
storage;,  summarization,  and  retrieval  of  the  ideas  and  data  of  science  and 
technology..  The  report  of  the  President's  Science  Advisory  Committee  on  Science, 
Government,  and  Information  (1963)  has  stressed  the  essential  point  that  "the 
information  process  is  an  integral  part  of  research  and  development «  Research 
and  development  cannot  be  envisaged  without  canmunication  of  the  results  of 
research  and  development;  morever^  such  ccramunication  involves  in  an  intimate 
way  all  segments  of  the  technical  communityo  not  only  the  docuraentalists. 

"We  place  special  stress  upon  what  seems  an  obvious  point  because^,  in  the 
early  days  of  science.,  the  problem  of  communication  could  be  managed  casually. 
Each  individual  scientist  could  work  out  his  own  private  communication  system, 
suitable  to  his  own  needs,  and,  since  the  requirements  were  relatively  small, 
the  whole  matter  could  be  treated  rather  incidentally.  But  with  the  growth  of 
science  a  casual  attitude  toward  communication  can  lead  only  to  insufficient 
communication.  Scientists  individually,  technical  societies,  agencies  supporting 
research  and  development,  will  have  to  recognize  that  adequate  communication  no 
longer  comes  free.  Communication  cannot  be  viewed  merely  as  librarians'  work, 
that  is^  as  not  really  part  of  science.  An  appreciable  and  increasing  fraction 
of  science's  resources,  including  deeply  motivated  technical  men  as  well  as  money, 
will  inevitably  have  to  go  into  handling  the  information  that  science  creates"  (p.lA)' 

The  overriding  questions  of  the  design  and  scale  of  information  systems  can 
only  be  solved,  however,  with  better  knowledge  of  the  information  needs  of  the 
users.  Surveys  of  time  allocation  by  scientists  (Ackoff  and  Halbert,  1958)  and 
engineers  iniicate  that  over  50  per  cent  of  their  working  time  is  devoted  to 
communication  functions s  receiving,  preparing  and  disseminating  information. 
Information  handling  serves  two  principal  functions;   (1)  management;  i.e.  direction, 
coordination,  reporting,  and  control,  and  (2)  technical  problem  solution.  We  are 
concerned  primarily  with  the  second  function.  '\'7'i\HHR 
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Previous  studies  have  asked  researchers,  by  interview  or  questionnaire,  what 
knowledge  they  needed,  what  sources  they  made  use  of,  and  what  function  the  infor- 
mation served  (Brownson,  1960).  These  studies  have  been  analyzed  and  compared  in 
a  monograph  prepared  for  the  National  Science  Foundation  by  the  Bureau  of  Applied 
Social  Research  of  Columbia  University  (1960).  Other  work  has  dealt  with  the 
reading  behavior  of  scientists  (Hensley,  1962;  Hoyt,  1962;  Scott,  1962),  and  the 
flow  and  use  of  information  and  ideas  in  university  and  industrial  research 
(Bureau,  1958,  Rubenstein  and  Avery,  1959).  It  should  be  noted  that  the  judgment 
of  the  researcher  has  been  the  only  criterion  of  the  value  of  the  information.  In 
this  study,  it  is  proposed  that  an  objective,  external  evaluation  of  solutions  be 
related  to  information  handling  and  utilization. 

COMPARISON  OF  INFORMATION  SOURCES 

In  order  to  discover  instances  where  information  should  have  been  available 
to  a  research  group,  but  was  not,  instances  must  be  sought  out  in  which  the  same 
problem  is  attempted  by  two  or  more  research  groups.  In  this  way,  a  comparison 
can  be  made  between  the  information  sources  leading  to  specific  solutions.  Upon 
ccmpletion  of  the  problem,  an  investigation  will  be  made  to  determine  whether 
these  sources  were  mutually  available  and  if  not,  what  the  impediments  may  have 
been. 

Research  on  research  is  intrinsically  difficult  because  the  outcomes  of 
different  projects  cannot  be  directly  compared.  The  opportunity  to  obtain  replica- 
tion of  attempted  solutions  to  the  same  research  problem  is  rare.  Seldom  do  two 
or  more  groups  undertake  to  solve  the  same  problem  at  the  same  time  (same  state  of 
knowledge).  This  is  inevitable  because  any  research  study  attempts  to  perform 
a  unique  task.  If  the  problem  has  previously  been  solved  it  is,  by  definition,  not 
a  research  problem. 
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Cojnparative  studies  of  research  success  might  be  possible  in  a  university 
graduate  department  if  a  number  of  Ph«D«  candidates  were  assigned  the  same  thesis 
problem.  Although  this  occurs  in  architectural  design  competitions;,  there  are 
no  instances  in  scientific  research.  Replication  of  problem  solving  is  easily 
achieved  in  laboratory  experimentation;,  but  in  such  experiments  the  problems  have 
been  trivial  and  have  not  approximated  the  complexity  or  requirements  of  even  the 
simplest  research. 

Real -life  opport.unities  for  the  canparative  study  of  differing  approaches  to 
the  same  problem  have  never  been  exploited.  In  addition  to  whatever  adventitious 
instances  of  simultaneous  parallel  research  activity  may  exist.,  there  are  three 
well-defined  situations  satisfying  this  criterion^ 

lo  Government  contract  proposal  competitions »  Procurement  agencies  in  the 
government  usually  distribute  a  Request  for  Proposal  to  which  frcan  6  to  20  firms 
respond  with  proposed  solutions  to  the  stated  problemo 

2.  Government  parallel  contracts.  In  a  limited  number  of  instances  a 
government  procurement  agency  will  award  identical  contracts  simultaneously  to 
two  or  more  firms o  Usually  these  contracts  are  for  feasibility  studies  or  pre- 
liminary designs.,  but  this  phase  of  the  R&D  process  is  one  that  contains  many  of 
the  difficult  and  theoretical  components  of  the  total  effort. 

3.  Program  definition  contracts .  A  new  procedure  in  government  contracting 
is  to  award  to  two  cr  more  firms  a  short-term  contract  calling  for  preliminary 
study  and  design  of  large  complex  systems »  The  resulting  reports  are  evaluated 
for  their  technological  validity.  Such  contracts^,  of  which  there  are  only  a  very- 
few  at  present^  will  provide  an  excellent  opportunity  for  comparative  study  of 
research  performance. 

Preliminary  exploration  of  the  possibilities  in  these  parallel  effort  situa- 
tions has  been  carried  out  only  in  the  first;,  or  proposal  competition.,  situation. 
The  results  (Allen  and  Marquis^  1963a,  1963b)  have  been  sufficiently  interesting 
to  justify  extension  of  the  work  in  similar  situations. 
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PROPOSED  RESEARCH  PLAN 

The  second  of  the  possible  situations-government  parallel  contracts — has  been 
selected  for  the  next  phase  of  the  study.  Program  definition  contracts  represent 
a  resource  for  the  near  future. 

Each  of  the  major  government  procurement  agencies  (DOD;,  NASA,  etc.)  will  be 
visited  in  order  to  identify  the  past.,  present,  and  future  parallel  contracts. 
Information  will  then  be  obtained  both  fran  the  government  agencies  and  from  the 
laboratories  performing  the  research.  Preliminary  investigation  has  indicated  a 
willingness  on  the  part  of  government  agencies  and  industrial  firms  to  cooperate 
in  a  project  of  this  sort. 

In  each  participating  laboratory^  the  project  director  and  his  principsQ. 
research  associates  will  be  given  a  detailed  structured  interview  at  the  beginning 
and  at  the  completion  of  the  contract  work.  They  will  be  asked  to  maintain  a 
daily  record  of  the  man-hours  spent  in  securing  technical  information  and  of  the 
sources  and  functions  of  such  information.  Because  attempts  to  secure  complete 
logs  or  diaries  frcan  busy  researchers  have  not  been  entirely  successful  in  the 
past,  a  special  form  has  been  designed  for  this  purpose.  This  form  minimizes  the 
amount  of  time  required  for  completion  by  using  codes  to  identify  the  particular 
methods  of  acquisition  and  the  functions  for  which  the  information  was  sought. 
In  classifying  functions,  we  will  utilize  categories  based  upon  those  developed 
by  Dr.  Herbert  Menzel  (1958)  of  Columbia  University.  Data  will  thus  be  obtained 
on  the  various  sources  of  ideas s  previous  related  experience,  memory,  journals, 
handbooks,  texts,  company  reports,  government  publications,  technical  experts 
within  the  ccmpany,  outside  consultants,  informsQ.  contacts,  etc. 

In  order  to  follow  the  course  of  development  of  the  major  solutions  during 
each  research  project,  the  project  director  will  dictate  on  a  tape  recording 
machine  the  principal  problems  encountered  and  the  means  by  which  a  solution  was 
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soughtc  Key  researchers  will  be  asked  to  estimate  weekly  the  probahility  that 
each  of  several  alternative  solution  possibilities  will  become  the  final  solution. 
This  will  be  accamplished  through  the  use  of  Solution  Development  Records  which 
require  the  researcher  merely  to  encircle  the  appropriate  probability  estimates. 

Upon  completion  of  each  project^,  the  competing  laboratories  will  be  revisited 
and  detailed  interviews.,  based  upon  data  already  collected j,  will  be  conducted. 
Since  both  solutions  and  information  sources  will  be  known  at  this  time^  inter- 
views will  be  directed  to  determining  which  sources  contributed  to  differing 
solutions  and  whether  these  sources  were  mutually  available  to  both  research  teams. 
In  cases  where  the  source  was  mutually  available,  more  detailed  information  will 
be  obtained  regarding  factors  which  may  have  interfered  with  proper  use  of  the 
information  source.  The  tape  record  of  problems  encountered  and  the  Solution 
Development  Records  will  be  used  to  isolate  major  difficulties  and  sudden  changes 
in  the  design.  The  critical  incident  technique  can  then  be  used  to  ask  researchers 
exactly  what  happened  at  a  certain  point  in  the  project.,  whether  there  was  a  failure 
or  difficulty  in  obtaining  information;,  and  how  information  leading  to  major  changes 
was  obtained. 

SCHEDULE 

On  the  basis  of  preliminary  experience  it  is  estimated  that  it  will  be 
possible  to  study  about  15  firms  at  the  same  time.  Since  the  average  duration 
of  the  projects  is  six  months^,  we  hope  to  complete  studies  of  30  firms  on  12  parallel 
contracts  during  the  year. 
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